Skip to content

docs/install: add xCAT provisioner with rocky10/almalinux10 recipes - #2654

Open
viniciusferrao wants to merge 1 commit into
openhpc:4.xfrom
VersatusHPC:xCAT-stateless-el10-recipe
Open

docs/install: add xCAT provisioner with rocky10/almalinux10 recipes#2654
viniciusferrao wants to merge 1 commit into
openhpc:4.xfrom
VersatusHPC:xCAT-stateless-el10-recipe

Conversation

@viniciusferrao

@viniciusferrao viniciusferrao commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

'An corp' xCAT stateless guides for OpenHPC 4.x with EL10 systems.

Sponsored-by: VersatusHPC

@github-actions

github-actions Bot commented Jul 11, 2026

Copy link
Copy Markdown

📊 Build Order Analysis Results

Environment: AlmaLinux 10 Container
Total Spec Files Analyzed: 89
Status:Build order analysis completed successfully

📋 Build Order Output

📦 Build Order (84 packages)

➕ Show remaining 69 packages

Analysis performed by OpenHPC Build Order CI
Commit: f33a895f
Workflow: 34241058596

@github-actions

github-actions Bot commented Jul 11, 2026

Copy link
Copy Markdown

📦 Package Count Analysis Results

Environment: UBI 10 Container
Target Branch: 4.x
Version: OpenHPC 4.x
Overall Status:Analysis completed successfully

🏭 Factory Repositories

Status:Factory repositories analysis completed successfully

Release Base OS aarch64 x86_64 noarch Total
4.2 EL_10 54 95 12 161
4.2 openEuler_24.03 48 55 12 115
Total All 102 150 24 276

Analysis performed by OpenHPC Package Count CI
Commit: f33a895f
Workflow: 34241058605

@github-actions

github-actions Bot commented Jul 12, 2026

Copy link
Copy Markdown

🚀 CCache Statistics

Configuration 🐧 RHEL 🦊 openEuler
🖥️ x86_64 / gnu15 🟢 56.67% 🟢 71.11%
🖥️ x86_64 / intel 🟢 75.00% -
💪 aarch64 / gnu15 🟢 56.67% 🟢 56.67%
📊 Detailed Statistics

openEuler (aarch64)

Cacheable calls:     30 /  62 (48.39%)
  Hits:              17 /  30 (56.67%)
    Direct:          17 /  17 (100.0%)
    Preprocessed:     0 /  17 ( 0.00%)
  Misses:            13 /  30 (43.33%)
Uncacheable calls:   32 /  62 (51.61%)
Local storage:
  Cache size (GiB): 0.0 / 5.0 ( 0.00%)
  Hits:              17 /  30 (56.67%)
  Misses:            13 /  30 (43.33%)

openEuler (x86_64)

Cacheable calls:     45 /  93 (48.39%)
  Hits:              32 /  45 (71.11%)
    Direct:          32 /  32 (100.0%)
    Preprocessed:     0 /  32 ( 0.00%)
  Misses:            13 /  45 (28.89%)
Uncacheable calls:   48 /  93 (51.61%)
Local storage:
  Cache size (GiB): 0.0 / 5.0 ( 0.00%)
  Hits:              32 /  45 (71.11%)
  Misses:            13 /  45 (28.89%)

RHEL (aarch64/gnu15)

Cacheable calls:     30 /  62 (48.39%)
  Hits:              17 /  30 (56.67%)
    Direct:          17 /  17 (100.0%)
    Preprocessed:     0 /  17 ( 0.00%)
  Misses:            13 /  30 (43.33%)
Uncacheable calls:   32 /  62 (51.61%)
Local storage:
  Cache size (GiB): 0.0 / 5.0 ( 0.00%)
  Hits:              17 /  30 (56.67%)
  Misses:            13 /  30 (43.33%)

RHEL (x86_64/gnu15)

Cacheable calls:     30 /  62 (48.39%)
  Hits:              17 /  30 (56.67%)
    Direct:          17 /  17 (100.0%)
    Preprocessed:     0 /  17 ( 0.00%)
  Misses:            13 /  30 (43.33%)
Uncacheable calls:   32 /  62 (51.61%)
Local storage:
  Cache size (GiB): 0.0 / 5.0 ( 0.00%)
  Hits:              17 /  30 (56.67%)
  Misses:            13 /  30 (43.33%)

RHEL (x86_64/intel)

Cacheable calls:     72 / 316 (22.78%)
  Hits:              54 /  72 (75.00%)
    Direct:          54 /  54 (100.0%)
    Preprocessed:     0 /  54 ( 0.00%)
  Misses:            18 /  72 (25.00%)
Uncacheable calls:  244 / 316 (77.22%)
Local storage:
  Cache size (GiB): 0.0 / 5.0 ( 0.00%)
  Hits:              54 /  72 (75.00%)
  Misses:            18 /  72 (25.00%)

🤖 Generated from workflow run 34241058774

@github-actions

github-actions Bot commented Jul 12, 2026

Copy link
Copy Markdown

Test Results

 29 files  29 suites   53s ⏱️
 59 tests 53 ✅ 6 💤 0 ❌
105 runs  97 ✅ 8 💤 0 ❌

Results for commit f9c0647.

♻️ This comment has been updated with latest results.

@middelkoopt

Copy link
Copy Markdown
Contributor

I most likely will not be able to review to this until after PEARC and PTO (Mid Aug), but will try. Thoughts on integrating stateless and stateful into one doc? We do something similar in Warewulf for provision-to-disk.

@viniciusferrao

viniciusferrao commented Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

I most likely will not be able to review to this until after PEARC and PTO (Mid Aug), but will try. Thoughts on integrating stateless and stateful into one doc? We do something similar in Warewulf for provision-to-disk.

Stateful is not ready yet. I wanted to finish that for the next TSC meeting be not sure yet. Still testing the steps.

But the idea was to file another PR. If we will merge in a single one, I would need to redo it.

If that's the approach let me know that I will start the rework instead of finishing the stateful part.

@middelkoopt

Copy link
Copy Markdown
Contributor

The general idea is to merge the two recipes, not have two separate ones - and that would be best as a single PR. I don't quite understand why there would be two. If they really need to be two recipes (very different workflow) then that's fine - it just seems odd (I've not looked at the previous xcat or used xcat in the past). Also note that there will be considerable testing to get it to actually get it working on our testing environment so doing both at once would be good.

@github-actions

Copy link
Copy Markdown

A friendly reminder that this PR had no activity for 30 days.

@viniciusferrao

Copy link
Copy Markdown
Contributor Author

I need to get the stateful back on track. I procrastinated due to the cancellation of next meetings.

@github-actions github-actions Bot removed the stale-pr label Aug 21, 2026
@viniciusferrao
viniciusferrao force-pushed the xCAT-stateless-el10-recipe branch from 1155a79 to 68b41e2 Compare September 7, 2026 02:17
@viniciusferrao viniciusferrao changed the title docs/install: add xCAT (stateless) provisioner with rocky10/almalinux… docs/install: add xCAT provisioner with rocky10/almalinux10 recipes Sep 7, 2026
@viniciusferrao
viniciusferrao force-pushed the xCAT-stateless-el10-recipe branch from 68b41e2 to 6ac4ba1 Compare September 7, 2026 02:32
@viniciusferrao

Copy link
Copy Markdown
Contributor Author

@middelkoopt At first I wasn't though it was a good idea, however I was able to merge both stateless and stateful in a single recipe. It required some replacement of information but it looked fine in my opinion, in chapters 5 and 7.

In fact it's way better now.

@viniciusferrao
viniciusferrao force-pushed the xCAT-stateless-el10-recipe branch from 6ac4ba1 to d7f638d Compare September 7, 2026 05:28
Adds xCAT to the Markdown recipe system, covering both of its provisioning
modes in a single guide. The mode is chosen at run time by ${xcat_mode} in
input.local rather than at document build time, so one guide and one
recipe.sh serve both, matching how the Warewulf recipe handles
provision-to-disk:

  stateless  compute nodes boot a RAM-resident image built in a chroot on
             the head node
  stateful   a base OS is installed to each node's local disk, then OpenHPC
             components are added to the running nodes

A "Select Provisioning Mode" section defines shell functions -- compute_exec,
compute_install, compute_group_install, compute_upgrade, compute_clean -- that
act on the image chroot when stateless and on the running nodes via xdsh when
stateful. The shared customization chapters call those helpers and are
otherwise untouched, so the InfiniBand, Omni-Path, memlock, GPU, syslog,
oneAPI and admin-tool sections are written once rather than once per mode.
Functions rather than variables because ${CHROOT} is only known after genimage
runs, and a function re-resolves it at call time.

Mode-specific steps live in templates/provisioner/xcat/ as stateless-* and
stateful-*, gated with ohpc_if so the generated script takes one path. Both
forked chapters group that work under a single heading -- "Stateless
Provisioning"/"Stateful Provisioning" and "Stateless Deployment"/"Stateful
Deployment" -- following the same aggregator-owns-the-grouping-header pattern
as "Cluster Admin Tools", so a reader follows one group and skips the other
outright. Nothing mode-specific sits outside those headings.

The deploy chapter starts Slurm on the head node first. Stateless nodes boot
afterwards and their slurmd reaches the controller on the first try, while the
already-running stateful nodes are then told to start theirs.

Details worth noting for review:

- The mode is a value, not a boolean. Every other input.local flag named
  enable_* turns an optional feature on or off, but this selects a mandatory
  mode, so it is spelled xcat_mode=stateless|stateful rather than a lone
  enable_stateful with no counterpart. It fails closed: anything else aborts,
  so a half-configured input.local cannot quietly netboot a fleet meant to
  install to disk.
- Image lookup refuses to guess. A head node holding images for several
  distributions or releases matches more than one, so the recipe stops unless
  exactly one matches or ${xcat_osimage} names it, and the chosen image is
  checked against the mode by its provmethod before anything is modified.
- The stateful wait requires every node to positively report a finished boot
  and to have lost a marker written to /run before the reset, so neither an
  unreachable node nor the system being replaced counts as ready. It gives up
  after ${install_timeout} rather than waiting forever on a failed install.
  It proves a reboot, not a successful install: a failed PXE attempt falling
  back to the existing disk would pass. Proving the install would need a
  per-run token written from the kickstart.
- The upgrade appendix spells out xdsh and --installroot commands per mode and
  re-derives ${CHROOT}, because the helper functions above exist only in the
  installing shell.

Both modes were deployed end to end on AlmaLinux 10.1 with xCAT 2.18.2 and two
compute nodes: the recipe ran start to finish in each, nodes provisioned to
tmpfs and to local disk respectively, Slurm showed both nodes idle, and a
two-rank MPI job ran across them.

Signed-off-by: Vinícius Ferrão <ferrao@versatushpc.com.br>
@viniciusferrao
viniciusferrao force-pushed the xCAT-stateless-el10-recipe branch from d7f638d to f9c0647 Compare September 8, 2026 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants